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Abstract 

This  thesis  is  a  software  design  and  implementation  of  the 
rear-end  for  the  Virtual  Information  Facility  of  the  INFOPLEX 
data  base  computer.  It  is  part  of  a  major  effort  to  develop  a 
software  simulation,  so  called  a  Software  Test  Vehicle,  STV  , 
for  the  underlying  architecture  of  INFOPLEX. 

INFOPLEX  is  a  hierarchical  architecture  for  data  base  com¬ 
puters,  based  on  functional  decomposition  of  data  base  oper¬ 
ations.  It  is  a  current  research  project  of  the  Information 
Systems  Group  at  M.I.T.'s  Sloan  School  of  Management.  Within 
the  INFOPLEX  architecture,  a  functional  hierarchy  of  informa¬ 
tion  management  functions  is  built  on  top  of  a  storage 
hierarchy  of  information  storage  functions.  These  two  inde¬ 
pendent  hierarchies  are  further  divided  into  many  sub-levels, 
each  of  which  is  devoted  to  a  more  specific  function  of  data 
base  activities. 
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The  virtual  information  facility  is  a  single  level  of  oper¬ 
ations  situated  within  the  functional  hierarchy.  It  supports 
the  use  of  virtual  information,  a  virtual  entity  based  on  pro¬ 
cedural  relationships  and  derivations  from  physically  recorded 
data.  Upon  completion,  this  facility  will  be  integrated  within 
the  current  implementation  of  the  the  STV  for  the  INFOPLEX 
functional  hierarchy  which  lacks  the  support  for  virtual 
information  processing. 

Thesis  Supervisor:  Professor  Stuart  E.  Madnick 

Sloan  School  of  Management,  M.I.T. 
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1.0.0  INTRODUCTION 


INFOPLEX  DATA  BASE  COMPUTER  is  a  current  research  project  of 
the  Information  Systems  Group  at  M.I.T.'s  Sloan  School  of  Man¬ 
agement.  It  proposes  a  new  architecture  whose  objectives  are 
to  provide  substantial  improvements  in  information  management 
performance  over  conventional  computer  architectures,  and  to 
provide  highly  reliable  support  for  very  large  and  complex  data 
bases. 

1.1.0  I NFOPLEX  OVERVI EW 


Progress  of  modern  society  has  put  increasingly  more  new  and 
challenging  demands  upon  the  capability  and  performance  of 
information  storage,  retrieval,  and  management.  Conventional 
computers,  whose  architecture  is  designed  primarily  for  compu¬ 
tational  objectives,  are  not  suited  to  meet  the  requirements  of 
these  new  demands.  Efforts  have  been  made  in  four  different 
areas  to  build  computer  systems  which  will  suit  our  information 
needs  today,  and  in  the  future:  (1)  new  instructions  through 
microprogramming,  (2)  intelligent  controllers,  (3)  dedicated 
computers  for  data  base  operations,  and  (4)  data  base 
computers.  INFOPLEX  is  a  research  project  belonging  to  the 
fourth  category. 
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1.1.1  CONCEPT 


INFOPLEX  employs  the  concept  of  hierarchical  decomposition 
which  organizes  information  management  functions  into  a  func¬ 
tional  hierarchy,  and  the  physical  memory  management  functions 
into  a  storage  hierarchy  (Madnick  78);  both  hierarchies  con¬ 
sist  of  many  independent  levels  of  operation,  each  of  which 
supports  a  different  set  of  information  or  storage  management 
functions  through  the  use  of  multiple  microprocessors . 

1.1.2  I NFOPLEX  ARCHITECTURE 

As  stated  previously,  INFOPLEX  is  an  architecture  for  data 
base  computers  based  on  hierarchical  decomposition.  A  func¬ 
tional  hierarchy  of  information  management  functions  is  built 
on  top  of  a  hierarchy  of  information  storage  functions.  Both 
hierarchies  are  further  divided  into  many  functionally  inde¬ 
pendent  levels  of  operation,  each  of  which  is  to  be  supported 
by  a  set  of  micro-processors  operating  in  parallel  with  one 
another.  A  global  Communication  Bus  coordinates  inter-level 
transmission  of  data.  This  hierarchical  architecture  exploits 
the  advantages  of  functional  modularity  of  operations,  and  of 
parallel  processing  of  micro-processors  to  systemize  data  base 
activities  and  to  achieve  a  prescribed  level  of  efficiency.  A 
graphical  illustration  of  this  architecture  is  presented  in 
appendix  A  (1.1.2) . 
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1.1.3  FUNCTIONAL  HI ERARCHY 

Current  architecture  of  the  functional  hierarchy  (Hsu  1982) 
with  respect  to  data  abstraction  consists  of  four  separate  lev¬ 
els:  (1)  external  level,  (2)  conceptual  level,  (3)  entity 
level,  and  (4)  internal  level.  A  part  of  the  conceptual  level 
is  a  virtual  information  facility  (Hsu  1982).  These  four  levels 
of  information  management  are  highly  independent  of  one  anoth¬ 
er,  and  each  is  responsible  for  a  different  but  necessary  phase 
of  information  processing  in  a  data  base  computer. 

1.1.4  RESEARCH  ISSUES 


Major  efforts  of  INFOPLEX  research  are  devoted  to  the  design, 
modeling,  and  evaluation  of  an  optimal  decomposition  strategy 
for  both  the  functional  and  memory  hierarchy  of  information 
management  and  storage  operation,  and  also  to  the  study  of  an 
associated  distributed  control  mechanism.  This  control  mech¬ 
anism  would  be  used  to  coordinate  the  activities  of  and 
inter-level  communications  within  the  hierarchies. 


1.2.0  THES I S  OB JECTI VE 

This  thesis  shares  a  joint  mission  with  a  concurrent  thesis 
by  Jameson  Lee.  The  two  theses  are  entirely  separate  in 
functionalities,  but  closely  related  and  dependent  upon  one 
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another  for  a  complete  software  simulation  of  the  virtual 
information  facility  on  the  INFOPLEX  data  base  computer  archi¬ 
tecture.  This  facility  would  incorporate  the  design  and  imple¬ 
mentation  of  two  sub-levels  of  the  INFOPLEX  functional  hierar¬ 
chy,  the  virtual  information  level,  and  an  user  interface  level 
which  is  tailored  for  the  use  of  virtual  information 
processing . 

Jameson  Lee's  thesis  is  responsible  for  the  fulfillment  of 
the  front-end  objectives  of  the  joint  mission;  his  objectives 
include  the  design  and  implementation  of  the  following; 

a)  A  data  base  language  to  support  virtual  information 

b)  A  finite  state  machine  to  parse  data  base  statements 
written  in  this  language 

c)  A  user-interface  tailored  to  the  use  of  virtual  infor¬ 
mation  . 

d)  A  processor  to  process  the  creation,  listing,  and  mod¬ 
ifications  of  virtual  definitions,  as  well  as  the  substi¬ 
tution  of  these  definitions  into  data  base  statements  in 
actual  use . 

This  processor  would  also  be  responsible  for  transforming 
data  base  statements  into  a  chain  of  tokens,  each  of  which 
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would  include  an  indicator  describing  the  classification 
of  the  token  according  to  a  prescribed  classification 
scheme . 

My  objectives  involve  the  implementation  of  the  following: 

a)  An  execution  tree  of  expressions  and  conditions  within 
retrieval  statements. 

b)  A  parser  which  implements  a  finite  state  machine  that 
takes  the  match-act ion-next_state  rules  as  input  from  the 
front-end. 

c)  An  entity  set  table  of  real  and  virtual  entity  sets 
within  retrieval  statements. 

d)  An  interface  with  the  entity  level  in  regard  to  the 
passing  of  values  of  real  attributes. 

e)  An  extraction  of  the  data  which  the  user  specified  in 
his/her  database  statements. 

f)  A  virtual  definition  catalogue  which  consists  of  two 
separate  dictionaries,  permanent  and  adhoc . 

The  combined  objectives  of  my  "back-end"  and  Jameson  Lee's 
"front-end"  would  fulfill  our  joint  mission  as  mentioned  ear- 
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lier,  namely,  to  construct  in  software  a  virtual  information 
facility  with  its  own  user  interface,  from  here  on  referred  to 
as  VIFI ,  the  Virtual  Information  Interpreter. 

1.2.1  BACKGROUND 

In  the  three  short  months  in  which  VIFI  was  develped,  we 
labored  and  wished  to  exhibit  a  certain  degree  of 
professionalism  in  its  design  and  implementation.  The  merits 
of  modular  programming,  of  innovative  algorithms,  of  perform¬ 
ance  efficiency,  of  functional  capabilities,  of 
user-friendliness  of  the  proposed  data  based  language,  of  pro¬ 
gram  organization  and  flexibility,  and  even  of  consistencies 
in  programming  style  were  evaluated  against  time  and  labor  lim¬ 
itations.  A  serious  attempt  was  made  to  incorporate  all  of 
these  characteristics  into  our  Virtual  Information 
Interpreter . 
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2.0.0  VIRTUAL  INFORMATION 


2.1.0  Concept 

The  concept  of  virtual  information  in  data  base  systems  has 
been  developed  and  examined  in  earlier  research  of  the  Informa¬ 
tion  Systems  Group.  Basically,  there  is  a  spectrum  of  the  kinds 
of  information  which  may  be  retrieved  from  a  data  base.  Along 
this  spectrum,  pure  data  occupy  an  extreme  on  one  end,  and  pure 
algorithms  occupy  the  extreme  on  the  other.  In  between  these 
two  extremes  are  the  information  which  may  be  derived  from  a 
combination  of  data  and  algorithms;  such  information  are 
dynamic  and  procedural  in  nature,  and  are  referred  to  as  Virtu¬ 
al  Information. 

2.2.0  CLASSIFICATION 

Virtual  information  may  be  categorized  into  three  major 
classes:  factored  facts,  inferred  facts,  and  computed  facts. 
Together,  these  three  classes  of  virtual  information  and  com¬ 
binations  there  of,  constitute  the  portion  of  the  information 
spectrum  between  the  two  extremes  of  pure  data  and  pure  algo¬ 
rithms  . 

2.2.1  FACTORED  FACTS 
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Factored  facts,  subsets  of  data  elements,  based  on  certain 
prescribed  conditions,  or  so  called  predicates,  of  attribute 
values,  are  often  very  valuable  in  structuring  information  in  a 
useful  manner.  For  instance,  if  a  certain  data  base  maintains 
records  of  weight,  hair  color,  and  salary  for  a  group  of 
employees,  it  may  be  useful  to  select  from  this  group  those 
individuals  who  share  a  certain  condition  on  their  attribute 
values,  such  as  having  black  hair,  making  a  salary  greater  than 
8  dollars  per  hour,  or  weighing  over  300  pounds.  It  is  impor¬ 
tant  that  users  of  information  should  be  able  to  access 
information  independent  of  the  particular  factoring  involved; 
this  would  imply  the  ability  to  support  multi-level  factoring, 
or  repeated  factoring  of  data. 

2.2.2  COMPUTED  FACTS 

Computed  facts  are  those  information  which  are  obtainable 

through  the  application  of  particular  computational  algorithms 

and  operators  on  data  or  groups  of  data.  These  operators 

include  arithmatic,  comparative,  boolean,  and  other  kinds  of 

functions.  In  the  very  least,  computed  facts  include  those  pure 

data  manifested  in  a  different  form,  with  a  different  unit  of 

measure,  or  an  alias  name.  For  instance:  a  user  may  define  a 

virtual  age  attribute  to  be  the  difference  between  the  current 

year  and  a  person's  birth-year,  a  virtual  rectangular  area 

attribute  to  be  the  length  multiplied  by  the  width,  or  an 
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attribute  value  in  the  unit  of  inches  to  be  12  times  the  attri¬ 
bute  value  in  the  unit  of  feet.  In  this  sense,  transformations 
between  different  units  of  measure  are  intrinsic  to  the  oper¬ 
ations  of  computed  facts. 

2.2.3  I NFERRED  FACTS 

Inferred  facts  pertain  to  implicit  relationships  which  the 
data  base  system  may  arrive  at  through  certain  levels  of  indi¬ 
rection.  In  other  words,  a  path,  although  indirect,  does  exist 
which  leads  to  the  desired  data  in  storage.  There  are  two  ways 
by  which  the  system  on  its  own  can  support  this  kind  of  virtual 
information.  The  first  method  is  by  an  exhaustive  search  of  all 
possible  paths,  and  the  second  is  the  application  of  a  certain 
degree  of  artificial  intelligence  to  deduce  a  viable  path  to 
the  target  data.  Well,  the  first  method  is  unbounded  in  comput¬ 
ing  time,  and  even  when  a  path  is  found,  it  may  not  be  the 
correct  path;  the  second  method  is  far  fetched  at  this  time. 
Therefore,  we  will  give  our  attention  to  a  different  but  compa¬ 
rable  set  of  inferred  facts  which  is  implementable ,  and  we  give 
it  the  name  Pseudo  Inferred  Facts.  Pseudo  Inferred  Facts  are 
exactly  the  same  as  inferred  facts  except  that  all  the  indi¬ 
rections  will  be  explicitly  designated  by  the  user.  With  this 
strategy,  exhaustive  search  is  not  necessary,  artificial 
intelligence  is  not  necessary,  and  the  specified  path  would 
always  be  the  designated  and  correct  path.  For  instance,  the 
Uncle  relationship  may  be  defined  as  the  application  of  the 
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Brother  relationship  after  the  application  of  the  Mother 
relationship. 

2.3.0  SPECIFICATION 

Users  of  information,  through  the  virtual  information  facil¬ 
ity,  define  their  own  working  environment  and  the  manner  in 
which  they  would  like  to  use  the  physical  and  underlying  data. 
Such  definitions  of  virtual  information  may  be  accomplished 
through  a  virtual  information  definition  language.  The  virtu¬ 
al  information  facility  would  accept  virtual  information 
definitions  and  their  modifications  in  the  definition 
language,  and  respond  to  virtual  information  retrieval 
requests  through  a  separate  virtual  information  retrieval  lan¬ 
guage  . 

2.4.0  MERITS 

There  are  several  major  merits  in  the  support  of  virtual 
information  in  a  data  base  system.  It  is  dynamic  in  nature 
because  its  definition  may  be  created,  deleted,  and  modified 
readily;  its  definition  applies  to  all  instances  of  data  where 
it  may  apply,  and  yet  there  is  but  only  one  copy  of  this  defi¬ 
nition  stored  in  the  system.  By  facilitaing  the  ease  of 
modification,  it  enhances  data  base  flexibility,  by  eliminat¬ 
ing  redundant  physical  records,  it  contributes  to  more 
consistent  data,  and  by  being  procedural  in  nature,  it  enhances 
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information  accuracy  through  the  delay  in  the  evaluation  of 
data  which  vary  over  time  or  other  changing  factors  until  their 
time  of  use.  These  kinds  of  merits  are  based  on  virtual  infor¬ 
mation's  association  with  procedural  relationships.  For 
instance:  the  stored  algorithm  for  computing  age  would  elimi¬ 
nate  the  need  to  update  the  age  attribute  day  by  day  if  it  were 
physically  stored,  and  would  be  applied  to  calculate  anyone's 
age,  thus  eliminating  redundancy  of  stored  information. 

Virtual  information  also  conserves  the  use  of  vast  amounts  of 
physical  storage.  It  makes  unnecessary  the  storage  and 
maintainence  of  those  information  which  may  be  derived  upon 
request.  This  raises  the  issue  of  Time/Space  trade-off,  which 
should  be  seriously  considered  when  deciding  which  kinds  of 
fundamental  data  are  or  are  not  to  be  physically  stored.  Deri¬ 
vation  upon  requests  will  have  the  added  cost  of  derivation; 
therefore,  those  information  which  will  be  used  many  times  and 
are  also  difficult  to  derive  may  be  the  best  kind  of  data  to  be 
physically  stored;  those  information  which  is  seldomly  used 
and  easy  to  derive  may  be  the  best  kind  of  data  not  to  be  phys¬ 
ically  stored.  Furthermore,  the  situation  is  made  even  more 
complex  as  we  realize  that  the  definitions  themselves  will 
require  the  use  of  physical  storage.  Thus,  it  wouldn't  be  an 
easy  task  to  decide  which  kinds  of  data  are  to  be  derived,  or  to 
be  actually  stored. 
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The  definition  of  virtual  information  on  a  per  user  basis 
would  simulate  an  entire  virtual  data  base  for  each  individual 
user.  Each  one  would  be  free  to  tailor  the  data  base  to  his  own 
preferred  view  or  use  through  the  virtual  information  defi¬ 
nitions.  A  particular  set  of  virtual  definitions  may  be  very 
useful  for  one  group  of  users,  and  another  set  for  another 
group  of  users.  In  this  sense,  each  one  has  gotten  a  data  base 
suited  for  his  own  use  while  not  affecting  anybody  else's  usage 
of  the  data  base.  A  logical  extension  of  this  scenario  is  to 
implement  access  control  mechanisms  such  that  users  may  estab¬ 
lish  a  controlled  sharing  of  sets  of  virtual  information 
definitions  with  one  another;  the  data  base  administrator  may 
monitor  all  such  sharing  to  prevent  unauthorized  access  to  a 
certain  set  of  virtual  information  functions.  However,  in  a 
scenario  as  such,  a  separate  catalogue  would  have  to  be  main¬ 
tained  for  each  and  every  user,  and  considerable  catalogue 
management  would  be  required.  Such  is  the  cost  for  this  indi¬ 
vidually  user-tailored  data  base  functionality,  a  secondary 
merit  of  the  use  of  Virtual  Information. 

2.5.0  APPROACH 

The  concept  of  virtual  information  leads  directly  to  a  func¬ 
tional  approach  to  data  bases.  A  virtual  information  facility 
would  be  treated  as  a  collection  of  functions,  and  retrieved 
data  would  be  regarded  as  functional  values.  Virtual  informa¬ 
tion  requests  correspond  to  function  invocations;  this  func- 
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tional  approach  to  information  readily  supports  procedural 
relationships  on  which  based  the  concept  of  virtual  informa¬ 
tion.  As  a  result,  a  virtual  information  facility  is  likely  to 
resemble  very  much  a  language  interpreter  which  accepts  func¬ 
tional  definitions  and  responds  to  functional  invocations  with 
specified  arguments. 


3.0.0  FUNCTIONALITIES 


There  are  numerous  functionalities  to  a  virtual  information 
facility,  each  of  which  may  be  implemented  to  a  varying  degree 
of  completeness.  Although  it  may  be  desirable  to  implement  all 
the  functionalities  there  are  wherever  possible,  it  may  be  too 
impractical  and  less  than  meaningful  for  the  initial  version  of 
the  implementation.  Thus,  we  have  not  implemented  the  One  Data 
Base  per  user  feature  of  virtual  information  capabilities 
which  we  have  described  in  the  previous  chapter.  Later 
portions  of  this  chapter  would  describe  the  functionalities  of 
virtual  information  which  we  did  implement;  surely,  not  all  of 
these  implementations  would  be  without  room  for  further 
refinement,  even  though  they  already  include  an  extensive  set 
of  virtual  information  capabilities. 

3.1.0  UNDERLYING  DATA  MODEL 

The  virtual  information  facility  lies  on  top  of  the  entity 
set  level  of  the  functional  hierarchy.  In  this  level,  the  data 
base  is  seen  as  a  network  of  entity  sets  and  their  attributes. 
Each  entity  set  may  have  a  varying  number  of  attributes,  some 
of  them  being  value  attributes  and  others  being  entity  attri¬ 
butes.  (Hsu  1980)  The  value  attributes  include  a  set  of 

attribute  values,  and  the  entity  attributes  represent 
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relationships  leading  to  other  entity  sets.  Illustrations  are 
found  in  appendix  A  (3.1.0). 

3.2.0  ACTIVE  WORKSPACE 


We  have  developed  an  active  workspace  which  incorporates  a 
line  editor  with  full  screen  display,  through  which  user  com¬ 
mands  may  be  issued.  The  workspace  consists  of  two  buffers,  an 
execution  buffer,  and  a  transaction  buffer.  The  transaction 
buffer  holds  many  data  base  statements  which  will  be  executed 
sequentially  when  the  transaction  buffer  is  executed.  The  exe¬ 
cution  buffer  holds  a  single  data  base  statement  and  will  be 
automatically  executed  when  a  data  base  statement  is 
completed.  A  number  of  buffer  commands  is  created  to  manipulate 
buffer  contents.  Examples  of  these  commands  as  well  those  of 
the  data  base  statements  are  illustrated  in  Appendix  A  (3.2.0) 
since  their  design  is  detailed  in  the  front-end  description  of 
the  VI FI . 

3.3.0  PERMANENTLY  DEFINED  VIRTUAL  INFORMATION 

Permanent  virtual  information  may  be  defined  through  the 
Define  statement.  Such  definitions  will  be  stored  in  a  global 
dictionary,  or  catalogue,  in  the  form  of  character  string,  and 
will  remain  there  until  explicitly  removed  or  over-written  by  a 
different  definition.  Examples  may  be  found  in  appendix  A 
(3.3.0) . 
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3.4.0  ADHOC  VI RTUAL  I NFORMATI ON 


Virtual  information  definitions  may  be  derived  for  only  the 
duration  of  a  single  transaction.  When  all  statements  within 
the  transaction  are  executed,  the  adhoc  dictionary  would  be 
erased.  Within  the  transaction,  adhoc  definition  may  be  cre¬ 
ated,  deleted,  as  well  as  modified  at  any  time.  With  this  fea¬ 
ture,  each  transaction  would  be  associated  with  a  catalogue  of 
its  own,  and  would  not  interfere  with  the  concurrent  activities 
of  other  transactions  executing  in  parallel.  At  this  stage,  we 
do  not  support  concurrent  transactions,  but  adhoc  definition 
capability  is  still  useful  in  the  principle  of  individual  envi¬ 
ronments  of  transactions  and  the  scoping  of  virtual  variables. 
Naturally,  the  permanent  dictionary  would  also  be  accessible 
from  within  each  transaction. 

3.5.0  NOTION  OF  A  TRANSACTI ON 

A  transaction  is  a  body  of  executable  statements  joined 
together  within  a  single  context.  This  context  is  provided  by 
the  adhoc  dictionary  associated  to  the  particular  transaction. 
A  transaction  is  created  within  the  transaction  buffer,  and 
will  remain  there  until  it  is  explicitly  over-written,  erased, 
or  executed.  Merits  of  this  transaction  concept  are  threefold: 
a)  a  group  of  statements  which  collectively  does  a  certain  task 
may  be  consolidated  to  exhibit  logical  unity,  b)  a  shared  con¬ 
text  may  be  created  and  maintained  for  each  transaction,  a  sign 
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of  transactional  modularity  and  independence  from  one  another, 
c)  the  execution  of  the  consolidated  operations  in  a  trans¬ 
action  may  be  put  off  until  a  more  opportune  moment,  by  which 
time  new  permanent  or  adhoc  virtual  information  definitions 
may  be  defined  either  to  supplement  or  to  replace  existing 
definitions. 

3.6.0  VI RTUAL  ATTRI BUTES 

Virtual  attributes  equated  to  the  results  of  computational 
algorithms  acting  on  available  data  or  of  designated  indirect 
references  may  be  explicitly  defined  through  the  Define  data 
base  statement.  This  feature  incorporates  the  support  for  Com¬ 
puted  Facts  as  well  as  for  Pseudo  Inferred  Facts.  For  instance, 
the  following  is  the  definition  and  usage  of  two  virtual  attri¬ 
butes,  income  and  ship-country,  a  computed  fact,  and  a  pseudo 
inferred  fact . 

Define  income  as  salary  -  expenses  ; 

Retrieve  (  ^teachers}  >  by  (  (v0(  -name ,  income )  ; 

The  foregoing  retrieve  statement  returns  two  vertical  col¬ 
umns  of  data.  The  first  column  being  teacher's  name,  and  the 
second  column  being  their  corresponding  incomes. 

Define  ship-country  as  company  {  country  (name) )  ; 

Retrieve  (  ^hip\)  by  (  -v-Oi  flame,  ship-country)  ; 
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This  foregoing  retrieve  statement  returns  two  columns  of 
data,  the  first  being  individual  ship  names,  and  the  second 
being  the  name  of  the  country  to  which  the  ship  belongs.  More 
examples  of  this  entity  set  is  in  appendix  A  (3.6.0). 

3.7.0  CONDITIONS  ON  REAL  OR  VIRTUAL  ATTRIBUTES 

Arbitrary  conditions  on  real  or  virtually  defined  attributes 
may  be  defined  by  INFOPLEX  users  as  the  shared  'condition'  on 
their  data  values  from  which  factored  facts  may  be  later  con¬ 
structed.  For  example: 

Define  old  as  age  >  70  ; 

Define  rich  as  assets  >  1000000  ; 

Retrieve  (  speople^  where  (rich  and  old)) 
by  (  [VO?  name ) ; 

The  foregoing  retrieve  statement  would  return  a  list  of  names 
of  those  people  whose  age  >  70  and  assets  >  1000000. 

3.8.0  VIRTUAL  ENTITY  SETS 

Aside  from  virtual  attributes,  we  also  support  a  basic  notion 
of  virtual  entity  sets.  We  recognize  two  kinds  of  virtual  enti¬ 
ty  sets: 
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a)  Union,  intersection,  or  cartesian  of  real  or  previously 
defined  virtual  entity  sets  based  on  their  real  and  virtual 
attribute  values. 

b)  Subsetting  of  real  or  virtual  entity  sets  based  on  certain 
conditions  on  their  real  and  virtual  attribute  values. 

For  instance: 

Define  ClassAB  as  (fciassAf  MU  (Name)  fclassBj j  f 

ClassAB  is  defined  as  the  result  of  a  multiple-union  opera¬ 
tion  on  entity  sets  ClassA  and  ClassB,  based  on  a  common  attri¬ 
bute  called  Name. 

Define  RichMen  as  ^Men^  where  (assets  >  1000000)  ; 

RichMen  is  defined  as  a  virtual  subset  of  the  set  Men,  based 

on  the  values  of  its  asset  attributes. 

The  complete  set  of  union  and  intersection  operators  as  well 
as  the  cartesian  product  operator  between  entity  sets  is  illus¬ 
trated  in  appendix  A  (3.8.0),  Also  included  are  details  of  the 
capability  to  specify  various  conditional  predicates  on  attri¬ 
bute  values. 
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3.9.0  GENERALI  ZED  MACRO  FACI LI TY 

Users  will  be  able  to  give  arbitrary  definitions  to  specified 
names  and  late  substitute  for  these  names  in  the  database 
statements.  In  this  sense,  the  define  statement  may  be  used 
not  only  to  define  virtual  attributes  and  virtual  entity  sets, 
but  also  random  definitions  that  may  seem  to  be  incoherent 
without  the  proper  context.  When  a  retrieval  statement  is  to  be 
executed,  each  word  within  the  statement  is  checked  against  a 
list  of  stored  definition  names;  any  word  that  matches  with  any 
definiton  name  would  be  replaced  from  the  dictionary  by  the 
definition.  Furthermore  the  words  in  the  definition  substuted 
would  likewise  be  substituted,  if  possible. 

3.10.0  EXTENDABLE  FUNCTIONALITIES 

3.10.1  USER  DEPENDENT  VI RTUAL  DEF I NI TI ONS 


This  particular  functionality  is  not  difficult  to  implement, 
but  it  may  be  unnecessary  at  this  stage  of  the  project.  It  sim¬ 
ply  would  require  a  separate  catalogue  for  each  user  which 
includes  an  access  control  list,  proper  search  rules  including 
default  situations,  and  adequate  coordination  and  control 
mechanisms  to  manage  the  various  catalogues.  It  would  increase 
the  cost  in  terms  of  time  and  space  efficiency.  Thus,  we  have 
not  included  this  functionality  in  this  version  of  virtual 
information  implementation.  Nevertheless,  if  circumstances  in 
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later  time  are  such  that  the  support  for  user  dependent  cata¬ 
logues  is  so  desirable  as  to  more  than  compensate  for  its  cost 
of  implementation,  this  functionality  may  be  added  readily.  At 
this  time,  we  can  simulate  this  by  tagging  all  definition  names 
with  the  unique  user  id  of  the  user  and  hence  make  the  name 
unique. 

3.10.2  INFERRED  FACTS  OF  UNDESIGNATED  INDIRECTION 

Inferred  facts  with  undesignated  indirection,  rather  than 
pseudo  inferred  facts  with  designated  indirection,  is  likely 
to  have  tremendous  costs  in  system  performance  whenever  it  is 
to  be  implemented.  As  previously  stated,  this  would  require 
either  an  exhaustive  search  or  a  certain  level  of  artificial 
intelligence,  both  of  which  require  large  amounts  of  resources 
in  computing  power,  storage  and  time.  Furthermore,  in  order  to 
verify  that  the  indirection  the  system  chooses  at  each  step 
along  the  way  is  correct,  the  user  has  to  monitor  the  computer 
decisions  interactively;  this  defeats  the  original  purpose  of 
not  having  the  user  to  designate  his  intended  path  of  indi¬ 
rection.  Thus,  it  seems  very  doubtful  that  this  functionality 
will  ever  be  implemented  unless  the  requirements  for  user  moni¬ 
toring  of  the  decision  process  is  somehow  eliminated. 
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4.0.0  OVERVIEW  OF  VIRTUAL  INFORMATION  SUB- SYSTEM 


4.1.0  DIVISION  BY  INFOPLEX  SUB-LEVELS 

The  virtual  information  sub-system  related  to  our  project 
extends  into  two  levels  of  INFOPLEX,  due  to  necessity  from  the 
modular  designer's  point  of  view.  The  interaction  between  the 
user  and  the  input  peripherals  of  the  sub-system  must  be  iso¬ 
lated  to  one  level,  seperate  from  that  used  to  process  the 
requests.  Since  part  of  our  sub-system  resides  in  a  level 
external  to  the  Virtual  Information  Level,  data  flow  must  be 
channeled  through  the  INFOPLEX  control  structure  data  bus  when 
passing  to  and  from  the  input  section  to  the  processing 
section,  in  a  proper  implementation.  A  diagram  of  our  entire 
sub-system  and  the  surroundings  is  in  appendix  A  (4.1.0) . 

4.1.1  USER  INTERFACE  LEVEL 

The  user  interface  level  currently  consists  of  a  user  inter¬ 
face  activity  coordinator  and  a  virtual  information  interface 
buffer.  When  completely  and  correctly  implemented,  the  user 
activity  coordinator  acts  as  a  switchbox  to  activate  either  the 
virtual  information  interface  buffer  or  a  real  information 
interface  buffer.  Likewise,  it  arbitrates  between  sending  the 

input  data  to  the  virtual  information  processor  or  the  real 
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information  process  in  the  Virtual  Information  Level.  In  addi¬ 
tion  to  placing  user  inputs  into  proper  storage,  the  interface 
buffer  alters  certain  characters  from  the  input  so  that  the 
lower  levels  may  safely  use  those  characters  as  delimiters. 

4.1.2  VIRTUAL  INFORMATION  LEVEL 

This  level  is  partitioned  into  two  sections,  one  of  which  is 
the  real  information  processor  that  creates  and  updates  entity 
sets.  We  are  not  concerned  with  that  portion  of  the  design. 
The  section  of  interest  is  the  virtual  information  processor. 
Here  we  have  a  virtual  information  activity  coordinator  that 
sequences  the  data  manipulators.  First  it  has  the  input  state¬ 
ments  cleaned  of  virtual  information  by  substitution  from  the 
dictionary;  it  then  converts  the  statements  into  tokens.  The 
tokens  translate  into  database  query  commands  when  they  are 
parsed  using  finite  state  transition  rules.  The  coordinator 
then  sends  some  queries  to  the  lower  INFOPLEX  entity  level  for 
processing.  The  returned  data  are  relocated  to  useable  storage 
and  processed  further  to  extract  the  exact  elements  which  the 
user  specified  by  his/her  command.  Control  then  goes  back  to 
the  User  Interface  Level. 

4.2.0  INTERNAL  INTERFACES 


To  avoid  unnecessay  conflicts  and  lack  of  determinism  in  data 
transfer  between  routines  internal  to  a  level,  the  coordina- 
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tors  in  the  two  levels,  User  Interface  and  Virtual  Information, 
are  capable  of  monitoring  all  data  movements.  This  is  a  merit 
of  the  horizontal  structuring  of  data  accessing  and  processing 
paths.  Each  interface  is  well  defined.  The  user  interface 
coordinator  transfers  character  buffers  in  both  directions  to 
the  buffer  processor.  The  buffer  can  be  then  altered  minimally 
and  sent  down  the  INFOPLEX  bus.  The  virtual  information  level 
activity  coordinator  sends  a  facsimile  of  the  character  buffer 
to  the  tokenizer  and  receives  a  list  of  tokens  in  return.  The 
virtual  level  coordinator  then  passes  the  tokens  to  the  parser 
and  receives  a  set  of  filled  tables  which  are  copied  into  a  sim¬ 
plified  version  to  be  used  as  a  query  request  to  the  lower 
levels.  The  filled  tables  become  the  main  means  of  trans¬ 
ferring  data  from  then  on.  There  presently  exists  a  dictionary 
which  should  be  later  fully  embedded  in  the  database  storage 
system.  In  other  words,  we  treat  internal  interfaces  very  much 
like  the  external  ones  which  attach  to  the  INFOPLEX  bus,  since 
we  want  modularity  and  simplicity  of  operation. 
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5.0.0  STORAGE  ARCHITECTURE  AND  MANIPULATING  METHODS 

Since  the  data  retrieval  requests  and  returns  are  based  on 
structures,  a  necessity  exists  to  clarify  what  is  involved  in 
accessing  these  structures  and  to  exhibit  the  kinds  of  data 
manipulation  that  can  be  performed  on  the  structures. 

5.1.0  I NTER-LEVEL  QUERY  STRUCTURES 

5.1.1  ENTITIES 


The  structure  ENTITY  consists  of  thirteen  sets  of  vertically 
arranged  entity  set  representations.  Each  entity  has  a  NAME, 
DEPTH  (number  of  values  in  a  single  occurrence  attribute), 
VES_FN  (entity  set  function),  WHERE  (discrimination  tree 
pointer),  N_PARENT  (entity  parents  pointers),  VES_PAR  (virtual 
entity  set  pointer  to  attribute  map  between  parent  entities  and 
child  entity),  and  a  set  of  fifteen  attributes.  Each  attribute 
has  a  VES_KEY  (designates  a  key  attribute  to  a  virtual  entity 
set),  CART_KEY  (designates  a  cartesian  key  attribute  to  a 
cartesian  virtual  entity  set),  SING_OCC  (designates  a  1:1  cor¬ 
respondence  attribute),  A_PARENT  (assigns  attribute  parent  if 
non-zero),  USES  (attribute  name),  and  LIST  (locates  all  values 
of  an  attribute) . 
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Real  entity  sets  are  created  from  the  information  available 
in  the  entity  set  representation.  An  abridged  version  of  ENTI¬ 
TY  is  made  and  sent  to  the  entity  level  of  INFOPLEX  and  is 
returned  with  attribute  values  linked  to  it. 

Virtual  entity  sets  are  created  from  real  entity  sets  or  from 
existing  virtual  entity  sets.  Whenever  VES_FN  specifies  that 
an  entity  set  is  virtual,  the  N_PARENTs  have  their  attributes 
mapped  by  the  N_MAP  attached  to  VES_PAR.  Based  on  the  proper 
relationship  amongst  the  keys  and  labels,  N_MAP  determines 
which  attributes  are  generated  by  which  existing  attributes; 
this  map  is  extremely  useful  for  future  linking  of  attributes 
across  entity  sets.  The  VES_FN  defines  the  particular  scheme 
for  creating  the  entity  set. 

5.1.2  ATTRI BUTE  VALUES  AND  LEVELS 

The  ATTRIB  structure  of  itself  is  very  simple.  It  contains 
one  attribute  value  and  the  level  number  of  that  value.  The 
inter-relations  of  the  ATTRI Bs  is  flexible  but  complex.  Attri¬ 
bute  values  are  linked  across  the  attribute  table  in  the  entity 
set  by  the  level  number;  such  a  set  is  always  copied  or  skipped 
as  one  since  the  correspondence  is  entire.  An  example  can  be 
seen  in  appendix  A  (5.1.2). 
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5.2.0  INTER-LEVEL  CONDITION  STRUCTURES 


5.2.1  EXECUTION  TREE 


XTREE  is  structured  such  that  it  represents  a  tree  with  nodes 
and  links.  Traversal  through  the  tree  gives  a  natural  sequence 
of  performing  operations  and  hence  a  correctly  built  tree  can 
be  used  as  the  control  structure  of  data  processing.  Each  node 
uses  the  LABEL  field  once  and  each  arc  of  a  node  uses  one  LINK. 
The  number  of  links  in  a  node  is  stored  in  CHILD  so  that  tree 
traversal  is  easily  achieved. 

5.2.2  TRANSITION  RULES  MACHINE 

MACH  stores  the  information  that  is  necessary  to  transform 
database  statement  tokens  to  query  structures.  In  a  full 
implementation  of  INFOPLEX,  MACH  should  be  totally  static, 
since  it  is  needed  for  any  inputs  to  the  VIFI .  The  control 
structure  itself  should  boot  up  the  entering  of  information 
into  this  table. 

5.2.3  ADDI TI ONAL  I NFORMATI ON  TABLE 

Useful  for  storing  more  information  than  is  allowed  by  a 
single  node  in  XTREE,  and  as  a  workspace  to  generate 
attributes,  XCHNGE  should  reside  in  the  VIFI  as  a  local  storage 
area,  in  the  same  repect  as  ENTITY. 
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5.3.0  EXTRA-LEVEL  QUERY  STRUCTURES 

5.3.1  QUERY  REQUESTER 

RETE_ARG  is  basically  an  extracted  version  of  ENTITY,  with 
control  structure  information  added.  Additionally,  it  has  a 
COND  sub-structure  to  store  simplifiable  conditions  for  lower 
level  processing. 

5.3.2  QUERY  I NFORMATI ON  RETURN 

RETE_RTN  is  essentially  the  storage  carrier  for  ATTRIB.  It 
contains  control  structure  information. 

5.4.0  PSUEDO  EXTRA-LEVEL  QUERY  STRUCTURES 

There  is  currently  storage  used  to  simulate  storage  that 
should  be  available  from  within  the  database  itself.  This  is 
due  to  the  fact  that  simulated  retrievals  are  more  easy  to  han¬ 
dle  and  check  for  errors. 

5.4.1  THE  DICTIONARY 


The  DICTION  table  is  split  into  three  sub-tables,  to  simulate 
a  number-of-lookups  CHECKer,  an  ADHOC  dictionary,  and  a  MAIN 
dictionary.  With  appropriate  tagging  and  catalogging  of  defi¬ 
nition  names,  the  main  dictionary  should  eventually  be  able  to 
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take  on  this  task.  Certain  amounts  of  security  restriction  can 
be  made  by  this  dictionary  interface  as  well. 
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6.0.0  SOFTWARE  IMPLEMENTATION 


The  software  described  is  fully  listed  in  appendix  B.  We 
used  PL/I  on  the  IBM  370  machine,  CMS  operating  system.  Only 
those  rear-end  routines  are  described;  front-end  implementa¬ 
tion  can  be  found  in  Jameson  Lee's  documentation.  By  relating 
to  the  overview  diagram  in  appendix  A,  one  can  see  the  corre¬ 
lation  of  the  software  routines. 

6.1.0  DICTIONARY 


The  dictionary  routine,  DCTNRY ,  is  used  only  to  simulate  the 
storage  of  data  in  the  main  database.  Since  some  necessary 
functions,  such  as  template  string  matching  or  variable  length 
strings,  are  not  yet  available  in  the  database,  a  more  powerful 
simulator  was  substituted.  Due  to  the  modularity  of  the 
system,  it  would  be  trivial  to  replace  this  with  the  real 
thing. 

Our  dictionary  currently  has  a  few  special  features.  It  can 
do  pseudo  parameter  passing  by  replacing  characters  in  the 
definition  by  characters  in  the  definition  name  matching 
string.  It  has  a  counter  that  prevents  unlimited  circular 
definitions.  Most  of  all,  it  has  a  defined  search  order  that 
consists  of  the  built-in  function  catalogue,  the  adhoc  dic¬ 
tionary,  and  the  main  dictionary.  The  adhoc  dictionary  and  the 
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counter  can  be  cleared  by  a  special  command  generated  by  the 
dictionary  user . 

6.2.0  MACHINE  DEF I NER 

DEFMCH  simply  reads  the  transition  rules  from  an  input  file 
into  a  table  which  can  be  accessed  more  readily  by  indexing. 


6.3.0  LANGUAGE  PARSER 

The  PARSE  routine  basically  interpretes  the  linked  list  of 
tokens  specified  by  TOKENS_PTR  and  produced  by  the  TKNIZE  rou¬ 
tine,  and  translates  this  list  to  a  request  format  to  be  later 
simplified  and  sent  to  the  lower  entity  level  for  processing 
and  retrieval.  It  does  the  interpretation  based  upon  a  set  of 
finite-state  transition  rules  that  have  been  input  by  the 
DEFMCH  routine  into  a  transition  rules  table  named  MACH.  The 
parameters  filled  by  the  parser  are  the  execution  order  tree, 
XTREE,  the  additional  information  exchange  table,  XCHNGE,  and 
the  unabridged  entity  set  query  table,  ENTITY.  Of  course,  the 
PARSE  routine  can  be  put  into  a  DEBUG  mode  to  follow  the  transi¬ 
tion  states  that  are  passed  given  a  set  of  tokenized  retrieval 
statement  and  the  transition  rules  table. 

The  main  data  storage  elements  in  PARSE  in  addition  to  the 
parameters  af orememt ioned  are  the  simulated  stacks,  STACK,  and 
the  virtual  entity  set  map  table,  VES.  The  first  two  stacks  are 
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used  by  the  finite  state  parser  in  a  push-down  automaton 
scheme,  in  which  they  are  respectively  labeled  as  the  operand 
stack  and  the  operator  stack,  whereas  the  third  one  is  used 
internally  and  exclusively  by  the  parser  to  track  the  addresses 
to  the  execution  tree. 

A  word  of  clarification  is  necessary  here.  In  the 
description  of  PARSE  to  follow,  the  term  'item'  refers  to  a 
multi-faceted  abstraction  of  data  in  the  sense  that  it  could  be 
anything  from  data  on  the  level  relevent  to  our  processing,  the 
same  data  with  internally  used  update  sources  or  labeling  tags 
imbedded,  to  internally  used  mapping  link  numbers.  Further¬ 
more,  the  data  being  processed  may  be  utilized  by  different 
levels,  such  as  the  finite  automaton  or  the  entity  set  request 
processor,  in  a  related  but  differing  way.  For  instance,  the 
the  item  '+:B'  ,  which  means  'plus  of  type  built-in  fuction'  ,  is 
the  composition  of  a  symbol  on  the  automaton  level  typed  in  by 
the  user  and  a  internally  generated  tag;  the  '+'  portion  is 
used  by  the  automaton  level  as  tokens  to  be  matched  and  by  the 
entity  set  retrieval  processor  as  the  operation  'plus'  to  be 
performed . 

PARSE  has  a  main  driver  loop  which  is  traversed  once  for  each 
transition  state  that  is  entered  by  any  particular  input  state¬ 
ment.  The  loop  coordinates  the  pairing  of  matches  that  fire 
according  to  MATCHING  and  the  actions  to  be  used  and  the  next 
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transition  state  to  be  entered  according  to  ACTING.  The  top 
level  sub-routines  of  PARSE  are: 

MATCHING  which  determines  whether  the  states  of  the  input 
linked  list  and  the  stacks  are  matchable  with  any  of  the 
required  conditions  for  firing  in  a  given  transition 
state . 

ACTING  which  assigns  the  actions  taken  by  the  current 
transition  state  to  arrive  at  another  state,  and  desig¬ 
nates  this  next  state. 

GETS  which  is  a  general  string  functions  that  breaks  an 
argument  item  into  two  parts  seperated  by  a  chosen  charac¬ 
ter  in  the  item. 

PUSH  which  pushes  an  item  onto  a  specified  stack. 

POP  which  pops  the  top  item  off  a  specifed  stack. 

CHANGE  which  alters  an  item  based  upon  special  characters 
in  the  item  selecting  sources  of  update. 

GENENT  which  takes  items  off  the  stacks  and  generates  an 
entity  request  into  ENTITY. 
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VIRTX  which  replaces  the  top  item  on  the  operand  stack, 
which  must  be  a  virtual  entity  abstract  map  number,  with 
the  corresponding  real  map  number. 

VIRTA  which  adds  a  new  entity  map  number  into  the  virtual 
entity  set  catalogue. 

GENNODE  which  takes  items  off  the  stacks  and  generates  a 
node  on  the  execution  tree,  with  the  appropriate  links  to 
other  nodes  in  the  same  tree.  The  link  to  the  new  node  is 
put  onto  the  operand  stack. 

GENATTR  which  puts  the  specified  attribute  name  into  the 
entity  set  retrieval  table,  with  the  appropriate  links  to 
other  attributes  in  the  particular  entity  set  of  interest. 
The  specified  attribute  name  is  converted  to  a  reference 
number  to  the  attribute  in  the  entity  set  table. 

EXCH  which  exchanges  the  top  item  on  the  operand  stack  with 
the  map  number  to  the  next  available  space  in  the  addi¬ 
tional  information  table  so  that  further  items  may  be 
appended  to  the  additional  information  table  as  part  of 
the  current  top  item.  A  tag  is  appended  to  the  top  item  to 
identify  what  type  of  additional  information,  indirection 
of  attributes  or  multiple  comparison,  is  at  issue. 
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ADDON  which  appends  additional  items  onto  the  most  cur¬ 
rently  addressed  space  in  the  additional  information 
table . 

Of  these,  MATCHING,  ACTING,  GENNENT,  GENNODE  and  GENATTR  are 
complex  enough  to  deserve  furthur  explanation. 

MATCHING  simply  decides  whether  or  not  a  certain  transition 
state  conditions  item  matches  the  current  existing  conditions 
of  inputs.  The  conditions  item  may  be  a  set  of  any  number  of 
conditions  each  of  which  if  matched  will  flag  the  match  as  pos¬ 
itive.  Each  condition  specifies  what  must  be  found  in  a  set  of 
sources  consisting  of  the  first  item  on  the  input  linked  list, 
and  the  top  of  stacks  of  the  first  two  stacks.  Furthermore, 
allowance  has  been  made  to  allow  translation  of  a  single  match¬ 
ing  item  to  a  multiple  list  of  possible  matching  items  to 
effect  storage  minimization  in  the  transition  rules  table. 

ACTING  processes  the  list  of  actions  found  in  the  transition 
rules  table.  It  takes  the  actions  one  by  one  and  distributes 
the  processing  over  several  loops.  Each  action  consists  of  an 
action  name  followed  by  arguments  whose  existence  and  typing 
vary  dependent  on  each  individual  action.  A  short  summary  of 
the  actions  consists  of: 

'push'  which  pushes  its  second  argument  onto  the  stack 
specified  by  the  first. 
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'pop'  which  pops  off  the  top  of  the  stack  specified  by  the 
first  argument,  using  POP. 

'del'  which  eliminates  the  first  item  on  the  input  linked 
list  by  advancing  TOKENS_PTR  and  deallocating  the  first 
token . 

'genent'  which  invokes  the  sub-routine  GENENT  to  generate 
an  entity  set . 

'virtx'  which  replaces  the  top  item  on  the  operand  stack 
with  the  real  map  number  indicated  by  the  item,  using 
VIRTX. 

'virta'  which  adds  a  new  entity  set  mapping  into  the  virtu¬ 
al  entity  set  catalogue,  using  VIRTA. 

'attwhr'  which  sets  the  appropriate  linkage  between  the 
entity  set  table,  ENTITY,  and  the  execution  tree,  XTREE. 

'gennode'  which  generates  a  node  on  the  execution  tree, 
using  GENNODE. 

'indx'  which  exchanges  the  top  item  on  the  operand  stack 
with  the  map  number  to  the  next  available  space  in  XCHNGE, 
using  EXCH  with  the  tag  for  indirection. 


'mulx'  which  exchanges  the  top  item  on  the  operand  stack 
with  the  map  number  to  the  next  available  space  in  XCHNGE, 
using  EXCH  with  the  tag  for  multiple  comparison. 

'addon'  which  adds  the  top  item  on  the  operand  stack  onto 
the  additional  information  table,  using  ADDON. 

After  all  actions  have  been  performed,  ACTING  designates  the 
next  transition  state  to  be  entered,  based  on  the  next  state 
entry  in  the  rules  table,  which  if  negative  would  indicate  a 
'subroutine  return'  found  on  the  operator  stack. 

GEMENT  does  the  generation  of  entity  set  requests  using  the 
ENTITY  table.  The  appropriate  items  must  already  exist  on  the 
stacks  to  allow  for  the  generation  and  hence  the  parsing  rules 
must  provide  for  the  set  up.  The  types  of  entity  sets  are: 

'r'  which  is  an  explicitly  user  defined  entity  set 
refering  to  a  real  entity  set  existing  inside  the 
database . 

's'  which  is  an  implicitly  created  entity  set  that  refers 
to  a  composite  entity  set  based  on  a  single  entity  set, 
with  additional  constraints  attached. 

'mu',  'mi',  'su',  'si',  'cs'  which  are  explicitly  user 
defined  entity  sets  refering  to  composite  entities  each  of 
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which  is  based  on  two  entity  sets.  The  specifics  of  each 
entity  type  are  described  elsewhere  in  this  document. 

The  necessary  attribute  keys  in  any  key  lists  are  catalogued  by 
GENENT.  An  entity  set  real  map  number  is  also  placed  on  the 
operand  stack  at  the  conclusion  of  a  generation  to  allow  proper 
linking  of  entity  sets. 

GENNODE  generates  nodes  on  XTREE  by  taking  the  top  item  on 
the  operator  stack  and  reserving  sufficient  number  of 
locations  on  the  tree  for  the  children  of  the  operator  of 
interest.  It  puts  the  link  numbers  to  these  locations  on  the 
internal  address  stack  and  proceeds  to  process  the  children  of 
the  operator,  which  may  be  other  non-terminal  or  terminal  oper¬ 
ators,  or  terminal  constants  or  variables.  The  operators  are 
represented  on  the  operand  stack  as  map  links  and  the  constants 
or  variables  exist  on  the  stack  as  themselves  with  their 
repective  taggings.  The  sub-routines  PUTX  and  GENATTR  are 
invoked  whenever  a  terminal  constant  or  variable  is  encount¬ 
ered. 

GENATTR  takes  items  pertaining  to  variables  about  to  be  put 
onto  the  execution  tree  or  about  to  be  recorded  as  keys  to  enti¬ 
ty  sets  and  places  the  decoded  information  into  the  entity  set 
table.  The  data  processed  indicate  the  name  of  the  attribute, 
any  parent  of  the  attribute,  and  whether  the  attribute  is  a 
key,  either  virtual  information,  cartesian,  or  both. 


Hence  we  have  described  the  operation  of  the  finite-state, 
push-down  automaton  parser  necessary  to  encode  the  user  inputs 
into  specifications  comprehensible  to  the  entity  level  for 
retrieval  purposes.  The  specifications  are  encoded  into 
pre-formated  tables. 

6.4.0  SIMPLIFICATION  OF  REQUEST 

The  SMPLFY  routine  minimizes  the  amount  of  data  flow  across 
the  boundary  from  the  virtual  information  level  to  the  entity 
level  by  abridging  the  entity  request  table,  ENTITY,  and  creat¬ 
ing  the  retrieval  set,  RETE_ARG.  Only  requests  for  real  entity 
set  information  exist  in  RETE_ARG.  In  addition,  the  execution 
tree,  XTREE  is  trimmed  by  the  removal  of  certain  easily  manipu¬ 
lated  conditions  and  the  placement  of  these  conditions  in 
RETE_ARG  for  processing  in  the  lower  levels  of  INFOPLEX. 
Before  any  of  this  is  done,  however,  SMPLFY's  task  is  to  assign 
all  attributes  correctly  into  the  appropriate  entity  sets, 
using  the  propagation  function  PRPGATE. 

PRPGATE  sets  up  the  proper  relationships  between  entity  sets 
and  attributes,  in  the  sense  that  attributes  used  in  any  entity 
sets  which  are  composed  from  other  entity  sets  must  exist  in 
the  parent  entity  sets  as  well.  An  attribute  map  table  is 
filled  for  each  composite  entity  set  to  make  the  attribute 
relationships  endure  though  the  retrieval  and  execution  phases 
of  the  data  processing.  All  attribute  names  and  parent 
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relationships  are  ensured  when  propagating  so  as  not  to  create 
conflicts  in  naming.  For  instance,  if  entity  B  is  based  on 
entity  A,  then  if  entity  A  has  attributes  X  ->  Y  ->  Z  (here,  we 
mean  that  X,Y,  and  Z  are  each  attributes  of  A,  with  X  being  the 
parent  of  Y  being  the  parent  of  Z)  and  entity  B  has  attributes  X 
->  U  ->  Y  ->  Z  then  entity  B,  after  propagation,  would  acquire 
the  new  attributes  U,  Y,  and  Z,  with  the  parent  relationship 
connected;  thus  B  would  have  two  attributes  by  the  name  Y  and 
two  by  Z. 

SEARCH  goes  through  the  trees  attached  to  each  real  entity 
set  and  attempts  to  simplify  their  structure  whenever  and  only 
whenever  comparisons  are  made  between  pairs  of  variables  and 
constants  and  which  are  at  most  joined  by  the  'and'  joiner.  The 
rationale  for  processing  thus  has  to  do  with  the  power  and 
efficiency  of  the  unary  level  underlying  the  VIFI .  The  trick 
that  SEARCH  uses  in  amputating  the  tree  involves  the  tagging  of 
nodes  so  as  to  only  prolong  those  branches  which  reach  unre- 
solvable  nodes  in  terms  of  simplification.  When  a 
simplification  is  found,  the  SETREL  sub-routine  is  invoked  to 
set  the  relationship  of  interest  into  RETE_ARG  for  computation 
at  the  lower  levels.  An  example  of  a  partially  resolvable  tree 
would  be  (name  *  'Bond'  or  id_num  =  '007'  and  job  =  'spy'), 
where  the  name  and  id_num  attributes  cannot  be  taken  out  of  the 
tree  but  the  job  attribute  can  because  it  is  joined  to  the  other 
conditions  by  a  simple  'and' . 
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Once  simplification  is  done,  the  control  of  the  virtual 
information  stage  must  be  temporarily  passed  down  to  the  entity 
level  for  request  processing  in  order  for  further  actions  to  be 
performable  in  the  virtual  information  level.  After  all, 
virtualness  has  to  be  based  upon  some  form  of  substance. 

6.5.0  CONVERS I ON  OF  STORAGE 

The  CNVERT  routine  is  a  simple  routine  to  transfer  data  from 
specialized  storage  necessary  to  the  INFOPLEX  control  struc¬ 
ture  to  storage  more  dedicated  to  the  internal  environment  of 
the  virtual  information  stage.  This  transfer  allows  uniform¬ 
ity  of  data  processing  in  the  VIFI.  Of  even  greater  signif¬ 
icance  is  the  amount  of  independence  and  flexibilty  the  VIFI 
achieves  from  the  external  environments.  Only  CNVERT  needs  to 
be  altered  should  any  interface  requirements  be  updated.  Once 
the  information  tags  and  attributes’  values  of  the  retrieved 
real  entity  sets  have  found  new  shelter,  CNVERT  proceeds  to 
eliminate  their  old  home. 

6.6.0  EXECUTION  OF  DI SCRI MI NATOR 

XECUTE  is  the  meat  of  the  data  processing  stage  of  the  VIFI. 
The  functions  which  it  performs  are  creating  virtual  entity 
sets  and  applying  additional  constraints  on  completed  entity 
sets  for  the  purpose  of  discrimination.  Here,  discrimination 
is  the  subsetting  of  the  entity  sets  based  on  the  user  speci- 
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fied  conditioning  clauses  which  were  not  simplified  and  sent  to 
the  entity  level.  The  sub-routines  of  XECUTE  are  divided  into 
two  components:  the  set  generation/condensing  routines  and  the 
subset  conditioning  routines.  They  act  on  the  retrieved  data 
which  have  been  attached  to  the  ENTITY  query  table  and  base 
their  actions  on  information  in  the  ENTITY  table  itself,  the 
execution  tree,  XTREE,  and  the  additional  information  table, 
XCHNGE. 


XECUTE  has  a  driver  loop  that  distributes  the  operations  on 
each  entity  set  in  turn.  The  set  generation/condensing  rou¬ 
tines  are: 


MUNI ON  which  builds  a  multiple  union  of  two  entity  sets  by 
copying  all  the  attribute  values  of  the  first  parent  and 
those  values  of  the  second  which  do  not  occur  in  the  first. 


MINTER  which  builds  a  multiple  intersection  of  two  entity 
sets  by  copying  only  those  attribute  values  of  the  first 
parent  that  exist  in  the  second. 


! 

i 

j 


SINGLE  which  condenses  an  entity  set  by  moving  all  attri¬ 
bute  values  of  the  entity  set  to  a  dummy  entity  set  and 
then  copying  back  only  those  values  whose  occurrence  is 
the  first  or  only  in  the  entity  set. 
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CRTESN  which  creates  a  cartesian  set  by  copying  multiple 
attribute  values  from  each  of  its  parent  entity  sets.  It 
copies  each  value  of  the  first  parent  the  number  of  times 
dictated  by  the  depth  (number  of  values  for  a  single  occur¬ 
rence  attribute)  of  the  second,  and  copies  sets  of  values 
of  the  second  parent  the  number  of  times  dictated  by  the 
depth  of  the  first. 

The  basic  sub-routine  with  which  entity  sets  are  copied 
is  COPYSET.  It  copies  an  entire  set  of  attribute  values 
from  a  specified  parent  entity  set  to  the  child,  with  spe¬ 
cial  care  in  regards  to  single  or  multiple  occurrence. 
Markers  are  set  by  the  caller  to  COPYSET  to  to  track  the 
et  of  attribute  values  to  be  copied  and  these  markers  are 
advanced  at  the  end  of  copying.  Level  numbers  are  used  to 
manage  the  copying  of  multiple  occurrence  items.  A  count¬ 
er  is  incremented  to  track  the  number  of  sets  of  values 
copied.  In  the  same  line  is  SKIPSET,  which  skips  a  set  of 
attribute  values  by  incrementing  the  abovement ioned  mark¬ 
ers  to  the  specified  parent  without  doing  any  copying. 
Hence  we  can  control  which  sets  of  attribute  values  are  to 
be  copied  to  the  child.  COPYALL,  on  the  other  hand,  just 
copies  all  values  from  an  parent  entity  to  a  child. 

Of  aid  to  set  generation  are  the  sub-routines  LOCATEKEYS 
and  IUMATCH.  The  first  sets  up  a  key  table  to  allow  easy 
access  to  entity  keys  for  the  purpose  of  determining 
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intersections  and  unions.  The  second  determines  if  two 
sets  of  attribute  values  are  in  fact  one,  according  to  the 
keys  assigned. 

The  subset  conditioning  routines  function  recursively  to 
traverse  the  parser-filled  execution  tree  in  order  to  perform 
the  user  specified  operations  on  the  attribute  values  of  the 
real  and  virtual  (composite)  entity  sets.  The  argument  passing 
between  the  recursive  calls  is  through  structural  duplicates 
of  the  linked  lists  containing  the  original  copies  of  the  val¬ 
ues.  The  main  recursive  sub-routines  are  as  follows: 

OPERATE  sequences  the  processing  of  the  nodes  by  identify¬ 
ing  each  node's  label  and/or  number  of  children,  and  sub¬ 
sequently  distributing  the  manipulation  of  the  child  or 
children  to  the  other  routines,  or  to  a  recursive  invoca¬ 
tion  of  itself.  A  node  of  no  child  is  either  a  variable, 
constant,  multiple  constant,  or  operator  of  no  argument; 
the  terminal  node  must  therefore  create  a  vertical  or 
unitary  list  to  return  to  the  tree  node  above.  A  node  of 
one  child  is  an  operator  of  one  argument  and  hence  must 
return  either  a  unitary  or  vertical  list,  depending  upon 
the  operator  type.  A  node  of  two  children  is  an  operator 
with  two  arguments  and  may  have  a  multiple  constant  as  a 
second  child.  The  GO_DOWN  routines  execute  the 
operations,  defined  by  the  labels  of  the  nodes,  on  their 
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children,  if  any  exist.  The  STR  routine  is  a  special 
string  operation  handler  which  is  also  recursive. 

STR  is  a  very  powerful  routine  that  handles  nodes  labeled 
'str'.  It  does  substring  operations  on  the  first  child  of 
the  'str'  node  whith  specifications  from  the  other  five 
children.  A  clearcut  description  of  the  ’str’  function  is 
in  Jameson  Lee's  description  of  the  front-end  of  the  VIFI . 
Essentially,  STR  uses  OPERATE  on  those  of  the  second, 
third,  fifth,  and  sixth  children  of  the  'str'  node  that  are 
relevent  to  the  type  specification.  The  fourth  child 
specifies  the  type  of  string  addressing,  absolute  or  rela¬ 
tive,  that  is  desired. 

The  support  routines  to  the  recursive  ones  above  are: 

GO_DOWNO  returns  a  unitary  list  containing  data  returned 
from  no  argument  operators. 

G0_D0WN1  checks  to  see  if  the  operator  of  one  argument  that 
it  handles  returns  unitary  lists  or  vertical  lists.  If  a 
vertical  list  is  to  be  returned,  it  goes  through  each  item 
in  the  list  to  evaluate  the  return  for  that  item. 

RET_UNI TARY  decides  whether  a  one  argument  operator  is 
defined  to  always  return  a  unitary  list.  It  also  evaluates 
such  operators  with  the  appropriate  operand. 
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UNARY  takes  a  one  argument  operator  and  its  argument  and 
computes  the  return. 

MULTEQ2  takes  the  '='  operator  and  its  two  arguments,  the 
second  of  which  must  be  a  multiple  constant,  and  evaluates 
the  results  of  the  first  augument  values  each  compared 
with  the  multiple  list. 

GO_DOWN2  determines  the  type  of  return  that  is  expected  of 
a  generic  two  argument  operator  and  proceeds  to  fill  out  a 
list  with  that  specification. 

BINARY  takes  a  two  argument  operator  and  its  arguments  and 
computes  the  return. 

CREATE_LI ST  creates  a  duplicate  list  of  the  values  of  the 
specified  attribute,  to  be  used  for  processing  in  upper 
nodes . 

CREATE_CONST  creates  a  unitary  list  of  the  single  constant 
passed  to  it.  It  also  can  create  a  multiple  constant  index 
list  which  is  used  to  address  a  multiple  constant  if  the 
terminal  node  on  which  it  acts  is  of  that  type. 

TMPLTE  is  a  function  that  can  be  used  to  do  template  match¬ 
ing  on  a  string,  hence  giving  the  user  this  additional  pow¬ 
er  in  value  searching. 
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DECRSET  is  used  to  decrease  the  number  of  values  in  an 
attribute  by  moving  an  entity  set  to  a  dummy  and  copying 
back  only  those  sets  of  attribute  values  that  have  a  corre¬ 
sponding  '  :true'  in  the  conditioning  list. 

The  routine  DISPOSE  is  useful  for  ensuring  no  storage  over¬ 
flow  since  it  is  invoked  whenever  unnecessary  storage  can  be 
deallocated. 

The  XECUTE  routine  that  is  used  to  discriminate  information 
to  the  level  specified  by  the  user  is  thus  described. 

6.7.0  FINAL  PROCESSING  OF  DATA 

Although  the  DOBY  routine  has  as  yet  not  been  implemented, 
due  to  unresolved  problems  regarding  output  interface,  it  is 
envisioned  that  this  routine  will  essentially  use  the  condi¬ 
tioning  sub-routines  of  the  XECUTE  routine  and  return  the  con¬ 
dition  lists  to  the  user.  Hence  it  follows  the  trend  and 
methodology  of  our  processing.  It  will  return  those  lists 
which  were  specifically  mentioned  by  the  user  and  tag  them  with 
the  labels,  which  could  be  of  type  virtual  information,  that 
should  be  extracted  by  the  tokenizer  in  the  earlier  stages  of 
processing. 

The  full  complement  of  software  for  the  rear-end  is  thus 
described. 
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7.0.0  DESIGN  CONSIDERATIONS  AND  OVERVIEW 


7.1.0  MODULARITY,  MODIFIABILITY,  AND  EFFICIENCY 

In  terms  of  modularity,  I  think  we  have  achieved  a  pretty 
good  record.  Our  design  strategies  concentrate  on  clean  inter¬ 
facing  of  routines,  with  clearcut  specifications  of  interfac¬ 
ing  parameters.  The  horizontal  control  structure,  regulated 
by  the  activity  coordinators,  provide  sensitive  debugging 
capabilities  not  available  in  vertical  programming.  The  man¬ 
ner  of  tokenizing  and  parsing  database  statements  allow  for 
changes  in  the  format  of  retrieval  requests  without  affecting 
the  inner  routines.  The  way  data  incoming  from  the  entity  lev¬ 
el  is  relocated  allows  easy  changing  of  interface  to  the  lower 
level.  In  general,  I  have  no  dissatisfactions  concerning  these 
i ssues . 

There  is,  however,  a  concern  I  have  regarding  the  complexity 
in  the  manner  in  which  certain  manipulations  of  data  are  per¬ 
formed.  Since  a  certain  amount  of  efficiency  is  desired  in 
processing,  a  lot  of  not-so-obvious  shortcuts  were  taken  to 
derive  results.  Hence,  documentation  is  required  on  the  level 
so  extremely  clear  that  it  becomes  a  drudgery.  Furthermore, 
certain  already  implemented  functionalities  will  not  be  used 

simply  because  not  enough  clear  documentation  is  available. 
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The  efficiency  of  the  INFOPLEX  structure  has  a  lot  to  be 
desired.  This  is  but  a  practical  matter  of  structural  design 
verses  kludgy  efficiency.  In  a  microprocessor  implementation, 
however,  i  am  sure  certain  structural  concepts  would  give  way 
to  efficiency  and  yet  maintain  that  its  integrity  factor. 

7.2.0  ADAPTABILITY  IN  OVERALL  INFOPLEX  ENVIRONMENT 

The  current  virtual  information  stage  is  in  concept  adapt¬ 
able  to  the  current  INFOPLEX  environment,  but  there  are  quite  a 
few  areas  in  which  conflicts  arise  because  of  lack  of  coordi¬ 
nation  of  effort  in  the  updating  of  the  system  as  a  whole.  Even 
though  modularity  is  desired,  certain  capabilities  cannot  be 
realized  unless  all  stages  of  the  system  support  those  capabil¬ 
ities.  With  parallel  microprocessor  processing,  the  speed 
consideration  in  the  inherently  slow  hierarchical  control 
structure  would  hopefully  diminish.  The  Test  Vehicle  can  only 
approximate  what  the  actual  system  can  do;  it  can  only  illus¬ 
trate  some  of  the  possible  ways  to  view  the  manner  data  ought  to 
be  managed. 

7.3.0  POTENTIAL  POWER  OF  VIRTUAL  INFORMATION 

I  personally  can  see  quite  a  lot  of  possibilities  in  this 
virtual  information  idea.  Our  current  implementation  is  some¬ 
what  restrictive  in  regards  to  some  of  the  desired  features, 
such  as  full  inferring  of  facts.  It  can  be  adapted  to  practi- 
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cally  any  situation,  however.  Unfortunately,  since  the 
re-implementation  of  programs  in  general  usually  is  easier 
done  than  the  updating  of  existing  ones  (I  program  enough  to 
know  this),  I  fear  that  our  package  is  only  a  transient  element 
in  the  progression  of  the  INFOPLEX  system.  As  it  is,  we  have 
put  into  VIFI  all  that  could  be  allowed  in  the  time  alloted,  and 
this  already  seems  far  enough  advanced  in  comparison  to  the 
other  components  of  the  system.  Hopefully,  sometime  all  the 
desired  features  can  be  put  together  and  INFOPLEX  can  see  the 
market . 
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8.0.0  CONCLUSION 


The  INFOPLEX  project  seems  to  be  a  promising  one.  The  virtu¬ 
al  information  concept  is  quite  realizable  using  various 
schemes  to  track  the  transformation  of  data.  According  to  the 
tests  we  have  done  to  generate  entity  level  requests  from  user 
inputs  that  we  would  consider  fairly  unrestrictive  in  terms  of 
power,  the  difference  between  'virtual'  and  'real'  information 
is  but  a  few  operations  to  connect  the  two.  Me  were  able  to 
construct  fairly  restrictive  entity  requests  when  the  user 
entered  retrieval  commands  sometimes  more  complicated  than 
those  found  in  programming  languages,  and  we  were  able  to 
resolve  the  commands  to  those  handled  by  the  lower  level,  for 
the  sake  of  efficiency,  and  those  handled  at  the  VIFI ,  for  the 
sake  of  computational  power  and  flexibility.  The  parser's 
approach  to  converting  user-input  commands  provides  a  natural 
extension  to  increase  the  capability  of  the  language.  As  it 
is,  the  user  can  already  define  a  lot  of  functions  based  on  the 
built-in  ones.  Me  can  easily  implement  such  things  as  data 
types  or  data  units  by  a  minimum  amount  of  hacking  because  we 
already  have  a  system  of  user  definitions.  However,  since  this 
is  only  the  prototype  of  the  virtual  information  processor,  we 
feel  that  what  we  have  is  more  than  adequate. 
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The  only  unfortunate  thing  is  that  we  were  not  able  to  fully 
test  out  the  processing  stages  of  the  VIFI,  since  we  lacked 
sufficient  test  data  and  time  to  establish  the  necessary  link¬ 
ages  to  the  other  components  of  INFOPLEX.  Hopefully,  our  VIFI 
will  eventually  be  integrated  into  the  Test  Vehicle,  and  I  am 
sure  it  will  fit  in  well  in  the  design. 
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